Improving physical appearance of humans has become a significant social priority in many countries, which explains the continuous rise in popularity of cosmetic products and skin care treatments. The skin care market is continuously growing, driven by a demand for customised products and by customers who are ready to pay for quantifying the results of their treatments by means of more and more sophisticated instruments. This has resulted in the development of in-vivo scanners for the analysis of skin. As opposed to the conventional methods of using a silicone rubber replica to examine the skin's topography, in-vivo scanners have the advantage of being less invasive and less prone to artefacts, for example as disclosed by J. Hatzis, Micron 35 (2004), 201-219.
A variety of in-vivo scanners are available on the market, employing a variety of techniques to acquire the images of the skin surface. One such scanner is manufactured by Moritex Corporation and described in U.S. Pat. No. 6,118,476. This instrument includes two polarised illumination systems having polarisation directions perpendicular to each other, and an analyser disposed on an optical path from the object to a CCD device. The analyser has a vibrating direction parallel to one of the illumination systems and perpendicular to the other one. The illumination system whose polarisation direction is parallel to the one of the analyser is sensitive to the surface state of the object, while the system whose polarisation is perpendicular to the one of the analyser provides information on the sub-surface layer. More recently, Moritex Corporation has patented another system in the US consisting of a main body and a detachable head and granted under number U.S. Pat. No. 7,151,956. An illumination light source (in the form of white LEDs) is placed in the main body together with an imaging apparatus. The detachable head is provided with optical fibres for guiding light from the illumination source to the object being imaged. Because optical fibres are much thinner than LEDs, according to the inventors, a more homogeneous illumination of the object can be achieved.
A device for measuring skin parameters similar to the one manufactured by ‘Courage+Khazaka electronic GmbH’ is described in U.S. Pat. No. 6,251,070. The apparatus consists of a casing containing optical recording means and light emitting means connected to a light source (a neon light tube). The aim of this patent is to increase the contrast of the images acquired by using light in a wavelength range between 350 and 400 nm. The instrument also provides means to measuring the level of humidity and sebum of the skin by means of a replaceable film placed at a distance from the optics. This film, initially opaque, becomes transparent as it absorbs the skin's secretions.
In U.S. Pat. No. 6,571,003, a method and apparatus for analysing a plurality of visual skin defects is described. In this US patent, the image acquisition system consists of a conventional digital camera. The digital image acquired is electronically analysed and defect areas are located. Subsequently, the system displays a second digital image, based on the first acquired digital image, identifying the defect areas by electronically altering the colour of a plurality of pixels corresponding to the skin defects.
In U.S. Pat. No. 6,907,193, the skin of a person is imaged by illuminating the skin with at least one light source, where the light emitted from the source is filtered using a polariser. The image is captured using a camera in a way that the angle formed by the light source, the skin, and the camera is from about 35 degrees to about 55 degrees. This arrangement is utilised to minimise the surface glare from the skin surface.
A hand-held device developed for dermatoscopy applications is described in U.S. Pat. No. 7,006,223. This device consists of two concentric rings of LEDs and a magnifying lens, through which the user views the patient's skin. A special arrangement of two polarisers allows switching between parallel-polarized and cross-polarized images to aid viewing internal structures as well as the skin surface.
All the techniques employed in the patents described above, image the skin surface in two dimensions. However it is preferable to provide images of the skin surface in three dimensions. Several techniques deal with the problem of recovering the three-dimensional shape of a surface known in the art. For example, the ‘binocular stereo’ technique is based on the acquisition of two images taken from different viewpoints. The depth of the surface is recovered by identifying corresponding points in the two images. This method has been successfully applied in cartography, but it suffers from several drawbacks. The main one is the determination of the corresponding features between two separate images, taken from different points of view. This requires the implementation of matching algorithms that result in additional complex computation.
U.S. Pat. No. 6,263,233 makes use of three-dimensional techniques for imaging the skin surface. A handheld microscope for the imaging of dermal and sub-dermal tissue is described. This instrument is based on the principle of ‘confocal microscopy’, and allows scanning the tissue at successive depths to provide images of vertical sections. By combining the optical slices, a three dimensional image of the tissue can be obtained.
‘Shape from shading’ (SFS) is a technique that computes the three-dimensional shape of a surface from the intensity variation in one image of that surface. SFS technique was proposed by B. K. Horn, The Psychology of Computer Vision, P. H. Winston ed., New York, 1975. Since describing surface orientation requires two variables, and measurements of brightness at a single point in the image provide only one variable, the problem of image reconstruction cannot be solved unless further assumptions are made. The assumptions made usually comprise knowing exactly the lighting condition and the surface reflectivity, as well as assuming that the surface is smooth and has a homogeneous texture. Furthermore, even if all these conditions are satisfied, the surface gradient cannot be uniquely determined for every pixel.
‘Photometric stereo’ was first proposed by Woodham, Optical Engineering 19 (1980) 139-144, and consists of varying the direction of incident illumination between successive images, while holding the viewing direction constant and taking at least two images for at least two different illumination directions. Woodham showed that three images taken for different illuminations are sufficient to uniquely determine both the surface orientation and the reflectance factor at each image point. An illustration of photometric stereo geometry is given in FIG. 1. A computer-based analysis is then used to determine surface orientation at each image point, usually by defining normal vectors of the surface for all these points.
Most photometric stereo methods use a technique when a multiple subsequent image frames are acquired by a camera when illumination conditions are changed by synchronous switching on/off three or more light sources with different position and illumination angle so that for each frame one source is illuminating the surface. Such a system is also described by European patent Publication number EP 1 814 083, assigned to Omron Corporation. Unfortunately the disclosed configuration does not eliminate the specular reflection by any means of optical configuration nor by any compensation during data processing. This means that for used Lambertian reflection model the system can be successfully applied only on relatively limited set of surface types with low specular reflectivity. For most purposes and most practical surfaces with non-negligible specular reflection, the measured data and the results are affected by significant errors.
It is known to use polarisation optics in various fields of optical imaging. One possible use is described in US patent publication number US2006/0239547, Robinson et al. The application does not use photometric stereo methods or its measurement configuration, neither does it perform surface shape reconstruction. Secondly it uses different property of light polarization—the dependency of the propagation depth on the polarisation state: by tuning this polarisation it selects the depth of the sub-surface layers from which the scattered light is detected on the sensor. As such this US publication does not aim to produce accurate 3D profiles of a surface and the methods described there cannot be used for detailed specification of the object shape.
In the photometric stereo measurements one of the main constraints for the accurate calculation of the surface tilt from reflected intensities is the use of appropriate optical reflection model to relate measured reflected intensities with the tilt angles of the surface. As the optical model varies depending on the surface type, a simple Lambertian reflection model is usually used if the optical properties of the surface are unknown or vary across the object surface. Normally the techniques which rely on Lambertian reflection model are prone to the errors caused by specular reflection on not fully diffusive surfaces. Some techniques try to eliminate the influence of specular reflection by the use of more than three illumination sources and the data burden with high specular component are excluded from the calculation. Such data are usually identified as the data with the maximum intensity from the set of all acquired frames for each individual pixel of the detector.
H. Saito, Y. Somiya and S. Ozawa, ACCV'95, 3 (1995), 348-352 describes a method to reconstruct the 3D shape of a skin surface replica using a modification of the photometric stereo technique. As explained above, in the conventional photometric stereo technique the gradient of the surface is uniquely determined by inverse analysis from three reflectance values of the three different light sources. However, the gradient cannot be determined if three values are not available due to the effect of shadow. To cope with such cases, Saito et al. define the evaluation function of the surface shape in consideration of the effects of shadow, and then reconstruct the shape by optimizing the evaluation using simulated annealing (SA). In other words, the shape of the surface is estimated by the iteration process. In each iteration, the estimated shape is evaluated by comparing the estimated shading images, which are synthesized from the estimated shape in the consideration of the effect of the shadow. The technique was later extended by A. Matsumoto, H. Saito and S. Ozawa, Electrical Engineering in Japan, 129 (3), 1391
Some other methods for skin topography measurement, such as interference fringe projection, optical triangulation and autofocuss can be also found in the literature, for example, J. M. Lagarde C. Rouvrais, D. Black, S. Diridollou, Y. Gall, Skin Research and Technology 7 (2001), 112-121 and J. L. Leveque, Journal of European. Academy of Dermatology and Venereology, 12 (1999) 103-114. However, many of these methodologies are problematic due to complexity of the instrumentation used and spatial constrains for the object positioning.
Some skin inspection systems use a number of individual detectors or array of the detectors to acquire the light directed to different angles after the reflection from particular point of the surface. Such systems can also be combined with multiple illumination sources such as the above mentioned US Patent Publication number US2006/0239547. Although the systems described in this application also comprises a translation system to scan through the area of interest, it does not provide means for full reconstruction of the surface shape and it does not use photometric stereo method for the surface shape recovery. On the other hand it describes a possibility to measure overall surface change in a photo-cell like arrangement with the use of collimated light source and a linear array of the detectors. The skin elevation is measured by detecting a shadow projected to the detector array. The application also describes the use of polarized light to tune depth of light propagation under the skin surface. It also uses of different light colors to further specify properties of the skin.
The techniques described above have been used in the most diverse fields, ranging from cartography to face reconstruction, to medical applications such as the endoscopic image of the stomach. However, they have not found widespread use due to a number of technical problems especially due to the different optical properties of inspected objects data must be processed in a different way for each different application.
An object of the present invention is to devise a means and method for accurate real-time three-dimensional imaging of a tissue surface, by effective suppression of the influence of specular reflection.
A further object of the present invention is to devise a means and method for obtaining spectral characteristics of tissue surface combined with the capability to reconstruct the three-dimensional image of the surface. A further object of the invention is to devise means and method for imaging tissue surface under conditions of more uniform illumination across the imaged area.